Scaling a three dimensional model using a reflection of a mobile device

ABSTRACT

A computer-implemented method for scaling a three dimensional model of an object is described. In one embodiment, first and second calibration images may be shown on a display of a mobile device. The display of the mobile device may be positioned relative to a mirrored surface. A reflection of an object positioned relative to the mobile device may be captured via a camera on the mobile device. A reflection of the first and second calibration images may be captured. The captured reflection of the object and the captured reflection of the first and second calibration images may be shown on the display. A reflection of the displayed captured reflection of the first and second calibration images may be captured. When the captured reflection of the displayed captured reflection of the first calibration image is positioned relative to the captured reflection of the second calibration image may be detected.

BACKGROUND

The use of computer systems and computer-related technologies continuesto increase at a rapid pace. This increased use of computer systems hasinfluenced the advances made to computer-related technologies. Indeed,computer systems have increasingly become an integral part of thebusiness world and the activities of individual consumers.

Computers have opened up an Industry of internet shopping. In many ways,online shopping has changed the way consumers purchase products. Forexample, a consumer may want to know what they will look like wearing orholding a product. However, online shopping has disconnected customersfrom the ability to try things out before buying. Online product pagesoften show a photograph of a model with or wearing a particular product,but users may desire a more accurate depiction of themselves in relationto various products.

SUMMARY

According to at least one embodiment, a method of determining a targetdimension on an object is described. In one embodiment, first and secondcalibration images may be shown on a display of a mobile device. Thedisplay of the mobile device may be positioned relative to a mirroredsurface. A reflection of an object positioned relative to the mobiledevice may be captured via a camera on the mobile device. A reflectionof the first and second calibration images may be captured via thecamera on the mobile device. The captured reflection of the object andthe captured reflection of the first and second calibration images maybe shown on the display. A reflection of the displayed capturedreflection of the first and second calibration images may be capturedvia the camera on the mobile device. When the captured reflection of thedisplayed captured reflection of the first calibration image ispositioned relative to the captured reflection of the second calibrationimage may be detected via the camera on the mobile device.

In some embodiments, an image of the object may be scaled based on thedetection of the relative positioning of the captured reflections of thefirst and second calibration images. A 3-D model of the object may begenerated and scaled based on the scaling of the image of the object. Animage of the object may be captured in relation to the detection of therelative positioning of the captured reflections of the first and secondcalibration images.

In one embodiment, a target dimension of the object may be determinedfrom the captured image of the object. The target dimension may be adistance between two points on the object. The object may be a humanface and the display may be positioned relative to the human face. Atleast one facial feature of the face may be detected. The facial featurespatially may be related to the target dimension. A detected aspect ofthe image may be related to a predetermined reference dimension inrelation to the detection of the relative positioning of the capturedreflections of the first and second calibration images. The firstcalibration image may include a machine-readable identification code.

A computing device configured to determine a target dimension on anobject is also described. The device may include a processor and memoryin electronic communication with the processor. The memory may storeinstructions that are executable by the processor to display first andsecond calibration images on a display of a mobile device, the displayof the mobile device being positioned relative to a mirrored surface,capture, via a camera on the mobile device, a reflection of an objectpositioned relative to the mobile device, and capture, via the camera, areflection of the first and second calibration images. The instructionsmay be executable by a processor to display the captured reflection ofthe object and the captured reflection of the first and secondcalibration images on the display, capture a reflection of the displayedcaptured reflection of the first and second calibration images, anddetect when the captured reflection of the displayed captured reflectionof the first calibration image is positioned relative to the capturedreflection of the second calibration image.

A computer-program product to determine a target dimension on an objectis also described. The computer-program product may include anon-transitory computer-readable medium that stores instructions. Theinstructions may be executable by a processor to display first andsecond calibration images on a display of a mobile device, the displayof the mobile device being positioned relative to a mirrored surface,capture, via a camera on the mobile device, a reflection of an objectpositioned relative to the mobile device, and capture, via the camera, areflection of the first and second calibration images. The instructionsmay be executable by a processor to display the captured reflection ofthe object and the captured reflection of the first and secondcalibration images on the display, capture a reflection of the displayedcaptured reflection of the first and second calibration images, anddetect when the captured reflection of the displayed captured reflectionof the first calibration image is positioned relative to the capturedreflection of the second calibration image.

Features from any of the above-mentioned embodiments may be used incombination with one another in accordance with the general principlesdescribed herein. These and other embodiments, features, and advantageswill be more fully understood upon reading the following detaileddescription in conjunction with the accompanying drawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate a number of exemplary embodimentsand are a part of the specification. Together with the followingdescription, these drawings demonstrate and explain various principlesof the instant disclosure.

FIG. 1 is a block diagram illustrating one embodiment of an environmentin which the present systems and methods may be implemented;

FIG. 2 is a block diagram illustrating another embodiment of anenvironment in which the present systems and methods may be implemented;

FIG. 3 is a block diagram illustrating one example of a scaling module;

FIG. 4 illustrates an example of a mobile device;

FIG. 5 illustrates an example of a mobile device showing the output ofits camera when the camera faces a mirrored surface;

FIG. 6 is a diagram illustrating a right side view of how reflectedimages may be viewed by a camera of a mobile device;

FIG. 7 is another diagram illustrating a right side view of howreflected images may be viewed by a camera of a mobile device;

FIG. 8 is another diagram illustrating a right side view of howreflected images may be viewed by a camera of a mobile device;

FIG. 9 is a diagram illustrating an overhead view of how reflectedimages may be viewed by a camera of a mobile device;

FIG. 10 illustrates an example of a mobile device shown with an objecthaving a target dimension;

FIG. 11 illustrates an example of a mobile device showing the output ofits camera where the calibration images are not aligned;

FIG. 12 illustrates an example of a mobile device showing the output ofits camera where the calibration images are aligned;

FIG. 13 illustrates an example of a mobile device showing the output ofits camera where the calibration images are not aligned;

FIG. 14 illustrates an example of a mobile device showing the output ofits camera where the calibration images are aligned and a calibrationimage is removed;

FIG. 15 is a flow diagram illustrating one embodiment of a method forscaling a 3-D model of an object;

FIG. 16 is a flow diagram illustrating another embodiment of a methodfor scaling a 3-D model of an object; and

FIG. 17 depicts a block diagram of a computer system suitable forimplementing the present systems and methods.

While the embodiments described herein are susceptible to variousmodifications and alternative forms, specific embodiments have beenshown by way of example in the drawings and will be described in detailherein. However, the exemplary embodiments described herein are notintended to be limited to the particular forms disclosed. Rather, theinstant disclosure covers all modifications, equivalents, andalternatives falling within the scope of the appended claims.

DETAILED DESCRIPTION

Online shopping may be especially unpopular in the eyewear industry.This may be due to the unique look of eyewear on each person's face.Some eyewear providers have websites that allow users to upload photosof themselves so that images of eyewear may be overlaid on their photos,but there are many drawbacks hindering user adoption of this shoppingexperience. One of these is the user's inability to see the eyewear frommultiple angles. Also, the scale of the photo may not match the scale ofthe eyewear image on the website, causing scaled composite images of theuser and the eyewear that may be misleading.

Some stores allow users to input measurements of their faces, but takingimportant measurements such as the pupillary distance between the eyesis difficult and unreliable without equipment from an optometrist and atrained practitioner taking the measurements. Currently, disappointingresults may occur due to issues with properly scaling depictions ofusers when virtually trying on or appearing with products beforepurchase. The emergence and increasing popularity of mobile computingdevices having high-resolution cameras and large displays providesopportunities to address many of these drawbacks.

By using three-dimensional (3-D) models of a user or object, a user mayperceive himself (or herself) more accurately, and by scaling an objectsuch as eyewear to the user's facial features, a much more accuratedepiction of the product can be given to the user. Mobile devices suchas tablet computers allow a shopper to take their measurements, in somecases, using nothing more than his device and a common mirror. Thesemeasurements can then be used to scale a 3-D model of the user's faceand produce an accurate depiction of the user's appearance as if theuser were physically wearing the clothing or eyewear.

The systems and methods described herein relate to the virtual try-on ofproducts. Three-dimensional (3-D) computer graphics may be graphics thatuse a 3-D representation of geometric data that may be stored in thecomputer for the purposes of performing calculations and renderingtwo-dimensional (2-D) images. Such images may be stored for viewinglater or displayed in real-time. A 3-D space may include a mathematicalrepresentation of a 3-D surface of an object. A 3-D model may beincluded within a graphical data file. A 3-D model may represent a 3-Dobject using a collection of points in 3-D space, connected by variousgeometric entities such as triangles, lines, curved surfaces, etc. Beinga collection of data (points and other information), 3-D models may becreated by hand, algorithmically (procedural modeling), or scanned, suchas with a laser scanner. A 3-D model may be displayed visually as atwo-dimensional image through a process called 3-D rendering, or used innon-graphical computer simulations and calculations. In some cases, the3-D model may be physically created using a 3-D printing device.

A device may capture an image of the user and generate a 3-D model ofthe user from the image. A 3-D polygon mesh of an object may be placedin relation to the 3-D model of the user to create a 3-D virtualdepiction of the user wearing the object (e.g., a pair of glasses, ahat, a shirt, a belt, etc.). This 3-D scene may then be rendered into a2-D image to provide the user a virtual depiction of the user inrelation to the object. Although some of the examples used hereindescribe articles of clothing, specifically a virtual try-on pair ofglasses, it may be understood that the systems and methods describedherein may be used to virtually try-on a wide variety of products.Examples of such products may include glasses, clothing, footwear,jewelry, accessories, hair styles, etc.

In the process of capturing an image and generating 3-D models of theuser and the object being worn, the object may be virtually scaled tobetter display the fit of the object on the user and to more accuratelyrender the appearance of the user's model wearing the object in virtualspace. In the process of scaling a model, a reference image may becaptured. The reference image may be used to identify a feature of theuser to compare to a reference dimension in the image. By making thecomparison to a reference dimension, measurements corresponding with thefeature of the user may be at least approximately determined in a mannernot heretofore known.

Having determined the measurements of features of the user, the 3-Dmodel may be more accurate in generating a realistic model of theobject, e.g., clothing, when it is placed on a model of the user. A moreaccurate rendering of the object on the user leads to a better userexperience when, for example, the 3-D model may be used to virtually tryon an article of clothing. The properly-scaled virtual model gives theuser a better preview of what the object will look like on the userwithout having to physically try it on. This can be advantageous insituations where the user may be shopping for an item without the itembeing immediately available, such as, for example, shopping via theInternet. While the appearance of an item may be communicated via imageson the Internet, the fit of an item on a user's unique face and head maybe represented when the object is matched to the user's uniqueproportions in the displayed view of the object, particularly when shownwith the user's head as part of the rendered scene.

Mirrored surfaces may be used in capturing and generating 3-D models ofthe user and the object being worn. Mirrored surfaces may include commonmirrors, including planar mirrors providing a 1:1 reflectedrepresentation of incident light. Mirrored surfaces may also includemultiple elements, such as, for example, a virtual mirror comprising anelectronic display showing the output of a camera pointing toward asubject in front of the display. In some embodiments, a virtual mirrormay include a mobile device (an iPad, for example) with a screen andfront-facing camera.

A virtual mirror may therefore simulate a common mirror in its functionby showing a reproduction of what is in front of the display. A virtualmirror may also provide additional features, including, for example,scaling at a non-1:1 ratio or horizontal or vertical reversal of thesubject. In some embodiments, the virtual mirror replicates a commonmirror as closely as possible. When scaling an image using a mirroredsurface having a ratio other than 1:1, a conversion ratio may be used toensure that measurements taken in a captured image are properly scaledto correspond with accurate proportions of the head of the subject.

Mobile devices may be used in the measurement and scaling of a 3-Dobject. A mobile device may be a mobile computing device, such as ahandheld computing tablet, smartphone, notebook or laptop computer, andrelated devices. A mobile device may typically include a camera and adisplay, but may include additional components. A camera of a mobiledevice may be located on the device to create a virtual mirror inrelation to the display. Thus, the camera's output may be shown on thedisplay of the mobile device with the camera being pointed at a subjectin front of the display. In some arrangements, the camera may beintegral to a bezel or other housing of the display, and in some otherarrangements, the camera may be separate from the display, such as, forexample, a webcam attached to the display or positioned near thedisplay. The camera may be in electronic communication with the displayso as to provide its output through the display.

A mobile device may include additional computing elements, such as, forexample, a processor and associated memory that are part of a tabletcomputer. In some arrangements the mobile device may be merely an outputof a distinct computing device, such as, for example, a monitor of adesktop computer. In yet further arrangements, a mobile device hascomputing elements and the display and camera may be in communicationwith a distinct computer, such as a desktop computer, where the distinctcomputer processes the input of the camera of the mobile device andsends the result to be output on the display of the mobile device. Insome arrangements the distinct computer may be accessed wirelessly bythe mobile device or through a connection to a network, such as a localarea network (LAN) or wide area network (WAN).

A display (of, e.g., a mobile device) may be an electronic display, suchas a liquid crystal display (LCD) or plasma screen or monitor. A displayof a mobile device may be capable of rendering images generated by acomputer, preferably in real-time, in order to provide an approximateappearance of a mirror to the user.

FIG. 1 is a block diagram illustrating one embodiment of an environment100 in which the present systems and methods may be implemented. In someembodiments, the systems and methods described herein may be performedon a single device (e.g., device 105). For example, the systems andmethods described herein may be performed by a scaling module 115 thatmay be located on the device 105. Examples of the device 105 includemobile devices, smart phones, personal computing devices, PDAs,computers, servers, etc.

In some configurations, a device 105 may include the scaling module 115,a camera 120, and a display 125. In one example, the device 105 may becoupled to a database 110. The database 110 may be internal to thedevice 105. In another embodiment, the database 110 may be external tothe device 105. In some configurations, the database 110 may includemodel data 130. Model data 130 may comprise data structures used in therendering and display of 3-D models, such as computer modeled eyeweardata or computer-simulated human facial feature data.

In one embodiment, the scaling module 115 may scale a 3-D model of anobject. In one example, scaling a 3-D model of an object enables a userto view an image on the display 125 that may be based on the scaled, 3-Dmodel of the object. For instance, the image may depict a user virtuallytrying-on a pair of glasses with both the user and the glasses beingscaled according to a common scaling standard.

In some configurations, the device 105 may be provided at a mirroredsurface along with an object having a measurable target dimension. Forinstance, a mobile computing device may be provided at a mirror alongwith the user's face. The target dimension may be defined as a distancebetween two points on the object, such as the distance between twofacial features when the object is a face. In some arrangements, thetarget dimension may be the pupillary distance on a face, as used inoptometry to define the distance between the subject's pupils.Determining the pupillary distance of a face may allow a more accuraterendering of glasses that fit the size of the user's face by scaling amodel of glasses to match the user's pupillary distance.

At the mirrored surface, the display 125 may be configured to show anoutput of the camera 120. With the display 125 facing the mirroredsurface and the camera 120 viewing the mirrored surface as well, thecamera 120 may be positioned to view the reflection of the display and areflection of the target dimension of the object.

When properly positioned, the scaling module 115 may show a depiction ofa first and second calibration image on the display 125 where thecamera's output of a reflected first calibration image (which is, forexample, a machine-readable identification code image) and the secondcalibration image (e.g., a bracket image) are aligned. For example, thereflection of the first calibration image as seen by the camera may bepositioned to fit within a bracket shape of the second calibrationimage. An exemplary construction showing the positioning of the device105 at a mirrored surface is described in further detail in connectionwith FIGS. 7-9.

To ensure proper alignment, the scaling module 115 may complete aprocess by which the alignment of the first and second calibrationimages may be confirmed by reading a machine-readable identificationcode (e.g., a quick response (QR) code) of the first calibration imagewhile it is shown inside the bracket of the second calibration image. Insome arrangements, the machine-readable identification code may begenerated after the type of mobile device is detected. For example, auser may input the type of device or type of display on the mobiledevice, and a QR code may be generated having predetermined on-screendimensions based on the size or resolution of the display of the mobiledevice. In other configurations, software of the device may recognizethe reflection of the device (e.g., its dimensions or identifyingcolors) and retrieve or generate a QR code corresponding to the deviceidentified. In one example, the user may access a website thatdetermined the type of device (based on browser session information, forexample) and provides a device-specific QR code that identifies thedevice so that a known size for the device may be determined. In somecases, the displayed QR code may be specifically formatted for thedisplay (e.g., screen and/or pixel configuration) of the mobile deviceso that the QR code is displayed at a known size. In this scenario, theQR code may itself (additionally or alternatively) be a referencedimension used in accordance with the systems and methods describedherein. By using a QR code as the first calibration image, it may beeasier to determine that the device is properly positioned, since aunique QR code would not be found in the background of a scene and wouldnot be mistakenly identified in the features of a face or other itemseen by the camera.

In some embodiments, the display may also be positioned adjacent to afacial feature (e.g., chin). This may help ensure that the camera has aview of both a reference dimension on the device and a target dimensionof a face.

The scaling module 115 may determine the target dimension of the objectby comparing the target dimension as seen by the camera to a referencedimension on the mobile device (e.g., the width of the first calibrationimage) as seen by the camera. In some arrangements, the comparison ofthe reference dimension and the target dimension may be facilitated byobtaining a reference image containing both dimensions as seen by thecamera and comparing the target dimension to the reference dimensionwithin the reference image. For example, for a reference dimensionhaving a known length, the distance between the endpoints of thereference dimension in the reference image may be directly compared tothe distance between the endpoints of the target dimension of theobject, and the target dimension may be proportionally calculated usingthese measurements in the reference image.

When necessary, a pattern recognition module may be accessed as part ofthe scaling module 115 to detect the endpoints of the target dimensionwithin the view of the camera. In some arrangements, the patternrecognition module detects head-related patterns such as, for example,facial features (e.g., eyes, nose, ears), or head shape or orientation.In some embodiments, the pattern recognition module finds non-facialfeatures such as, for example, dots on a page, letters, reference points(e.g., the horizon), corners, or shapes (e.g., the shape of the device105).

With the target dimension determined, the scaling module 115 may accessthe model data 130 of the database 110 to produce a scaled 3-Drepresentation of the model which matches the scale of the object inrelation to the target dimension. For instance, the model may beeyeglasses which may be scaled to fit a user's face in relation to theface's pupillary distance. In some embodiments, the model data 130 mayinclude one or more definitions of color (e.g., pixel information) forthe 3-D model of the object (e.g., user). In one example, the 3-D modelof the object may have an arbitrary size. In some embodiments, thescaled 3-D model of the object (as scaled by the systems and methodsdescribed herein, for example) may be stored in the model data 130. Insome cases, the model data 130 may include the image of the user holdingthe mobile device.

In some configurations, an image based on the scaled 3D model of anobject may be displayed via the display 125. For example, an image of avirtual try-on based on the scaled 3D representation of a user and a 3Dmodel of glasses scaled according to a common scaling standard may bedisplayed to the user via the display 125.

FIG. 2 is a block diagram illustrating another embodiment of anenvironment 200 in which the present systems and methods may beimplemented. In some embodiments, a device 105-a may communicate with aserver 210 via a network 205. Examples of networks 205 include localarea networks (LAN), wide area networks (WAN), virtual private networks(VPN), cellular networks (using 3G and/or LTE, for example), etc. Insome configurations, the network 205 may be the internet. In someconfigurations, the device 105-a may be one example of the device 105illustrated in FIG. 1. For example, the device 105-a may include thecamera 120, the display 125, and an application 215. It is noted that insome embodiments, the device 105-a may not include a scaling module 115.

In some embodiments, the server 210 may include the scaling module 115.In one embodiment, the server 210 may be coupled to the database 110.For example, the scaling module 115 may access the model data 130 in thedatabase 110 via the server 210. The database 110 may be internal orexternal to the server 210.

In some configurations, the application 215 may capture one or moreimages via the camera 120. For example, the application 215 may use thecamera 120 to capture an image of an object having a target dimensionand a reference dimension. In one example, upon capturing the image, theapplication 215 may transmit the captured image to the server 210.

In some configurations, the scaling module 115 at the server 210 mayoperate as described above and as will be described in further detailbelow. In one example, the scaling module 115 may transmit scalinginformation and/or information based on the scaled 3-D model of theobject to the device 105-a, such as, for example, pattern recognitionresults and related data. In some configurations, the application 215may obtain the scaling information and/or information based on thescaled 3-D model of the object and may output an image based on thescaled 3-D model of the object to be displayed via the display 125.

FIG. 3 is a block diagram 300 illustrating one example of a scalingmodule 115-a. The scaling module 115-a may be one example of the scalingmodule 115 illustrated in FIG. 1 or 2. In some configurations, thescaling module 115-a may obtain an image (depicting an object and areference dimension, for example) and a 3-D model of the object. In oneexample, the image may depict only a portion of the object and only aportion of the reference dimension. As noted previously, the referencedimension may have a known size. As will be described in further detailbelow, the scaling module 115-a may scale the 3-D model of the objectbased on the known size of the reference dimension. In someconfigurations, the scaling module 115-a may include a display module305, a calibration module 310, a measurement module 315, and a scaleapplication module 320. A measurement module 315 may also comprise apattern recognition module 325 in some cases.

A display module 305 may provide logic for the generation and output ofgraphics and images to a display (e.g., display 125), including imagessuch as the first and second calibration images. The display module 305may scale the on-screen dimensions of one or more of the calibrationimages to adjust the desired alignment position of the device in frontof a mirror, as described in further detail in connection with FIGS. 6through 9 below.

A calibration module 310 may provide logic for the detection of thealignment of the calibration images as seen by the camera when themobile device is properly positioned in front of a mirrored surface. Thecalibration module 310 may also analyze the images received by thecamera in order to detect whether the target dimension is in view priorto capturing an image or making a comparison of a reference dimension onthe device to the target dimension.

A measurement module 315 may provide logic for the measurement of thetarget dimension in comparison to the reference dimension. Thus, themeasurement module 315 may include instructions for conducting such acomparison, including, for example, detecting endpoints of the targetdimension and the reference dimension. In some embodiments, themeasurement module 315 may include a pattern recognition module 325enabled to detect patterns that allow the scaling module 115-a todetermine the areas or points seen by the camera that are detected andmeasured when these areas or points are a part of patterns (e.g.,features on a face). A measurement module 315 may also provide theability to calculate the comparison of the target dimension to thereference dimension and thus produce the final measurement needed forscaling a 3-D model to match the object's scale.

A scale application module 320 may obtain the measurement informationproduced by the measurement module 315 and, in some embodiments, thepattern recognition module 325, to produce the scaled 3-D model usingthe determined target dimension. For example, the scale applicationmodule 320 may obtain a measurement of the distance between the pupilsof the user after comparison of that distance to a reference dimension(e.g., the width of the first calibration image), and determine that a3-D model of a pair of glasses should be scaled to an appropriate sizeto match a model scaled to approximate the user's head. Thus, when themodel of the glasses is placed with the model of the user's head, a moreaccurate representation may be generated of how the glasses would appearto the user if he or she were to physically try them on.

FIG. 4 is a diagram 400 of a front view of a mobile device 405. Themobile device 405 may be the same device 105 as described in FIGS. 1 and2. The mobile device 405 (e.g., tablet computer) has a display 410 onwhich images may be shown. The mobile device 405 also has a camera 415,which is shown near the top edge of the device 405. In somearrangements, the output of the camera 415 may be shown on the display410. The camera 415 may be positioned in a central portion of a bezel418 of the device 405 as shown. For example, the camera 415 may beintegrally formed with a top central portion of the device 405. Acentrally-positioned camera allows the display 410 to better act as avirtual mirror when the output of the camera 415 is displayed centrallyon the display 410. In other embodiments, the camera 415 may bepositioned off-center on the device 405 or may be external to the device(e.g., a webcam or camera attachment) as described above.

The display 410 may show at least a first calibration image 420 and asecond calibration image 425. The display 410 may also sequentially orsimultaneously show the output of the camera 415. In some embodiments,the output of the camera 415 may be displayed behind the calibrationimages 420, 425 (see, e.g., FIGS. 5 and 11-14). In some of theseconfigurations, the calibration images 420, 425 may be at leastpartially transparent in the display 410, allowing at least part of theoutput of the camera 415 to be seen through the calibration images 420,425 displayed. For example, the second calibration image 425 may bepartially transparent throughout the entire image, or it may becompletely transparent at portions of the image, such as if it were abracket or frame allowing the output of the camera 415 appear to be“behind” the display of the image 425. When the calibration images 420,425 are at least partially transparent, the user may find it easier toorient the camera 415 and thus align the calibration images 420,425.

The first calibration image 420 may be displayed above the secondcalibration image 425 on the display 410. The first calibration image420 may be a machine-readable identification code, for example, a quickresponse (QR) code or a barcode. Alternatively, the first calibrationimage 420 may be another image that may be recognizable by a calibrationmodule 310 and measurement module 315. In some embodiments the firstcalibration image 420 preferably has high contrast, allowing thecalibration and measurement modules 310, 315 to more easily identify andverify the alignment of the first calibration image 420 with the secondcalibration image 425 when the output of the camera is simultaneouslyshown on the display 410.

In some instances, the object being measured by the device 405 may bedisposed above the camera 415. Here, the first calibration image 420 maybe preferably above the second calibration image 425 so that it may bepositioned closer to the object and the camera 415. In otherconfigurations, the first calibration image 420 may be displayedelsewhere on the display 410 to facilitate ease of obtaining ameasurement using the device 405.

The first and second calibration images 420, 425 may not overlap on thedisplay, at least in the portions of the images 420, 425 that areintended to align when the device 405 is positioned for obtaining ameasurement of an object. The first and second calibration images 402,425 may be positioned along a common line across the display 410, forexample, along a common central vertical axis running across the frontof the display. The diagram 400 shows the calibration images 420, 425positioned vertically in-line with the camera 415 and the rest of themobile device 405.

The second calibration image 425 may be a shape (e.g., a box or circle)or other image (e.g., a bracket, a series of dots, or a set ofcrosshairs). The second calibration image 425 may be a partiallytransparent version of the first calibration image 420. In someembodiments, the second calibration image 425 has smaller on-screendimensions than the first calibration image 420 (see, e.g., FIG. 4). Forexample, the second calibration image 425 may be a scaled version of thefirst calibration image 420. The second calibration image 425 may haveat least partial transparency, as described above, but opaque images orimages with adjustable transparency may be part of other alternativeembodiments.

FIG. 5 is a diagram 500 illustrating one example of a device 405 showingthe output of its camera 415 on its display 410 when the camera 415 maybe facing a reflective, mirrored surface. The display 410 shows thereflected mobile device 505 having a reflected first calibration image510 and a reflected second calibration image 515. The reflected mobiledevice 505 may be seen with the reflected first calibration imagealigned with the second calibration image 425, which may not be areflection. The output of the camera 415 may be seen on the display 410simultaneously with the first and second calibration images 420, 425.Thus, the display 410 may present reflected and non-reflected versionsof first calibration images 510, 420 and reflected and non-reflectedversions of second calibration images 515, 425 simultaneously.

Notably, in these embodiments the reflected device 505 may not be areflection directly created on the display 410 (e.g., by the surface ofthe display 410 being a polished surface). Instead, the reflected device505 may be the camera's 415 view of the reflection of the device 405 inthe mirrored surface, and that view may be shown on the display 410. Ifthe user covered up the camera 415, the reflected device 505 may not beseen on the display according to these embodiments. However, otherembodiments allow actual reflections on the display to be used inaligning a mobile device.

If the camera 415 changes direction, the reflected device 505 changesposition on the display 410. Therefore, the reflected device 505 may notbe seen on the display 410 if the camera 415 is not facing thereflection of the device 405. In order for the alignment of FIG. 5 tooccur, the camera 415 may be angled to cause the reflected firstcalibration image 510 align with the second calibration image 425.

FIG. 6 is a diagram 600 illustrating a right side view of an object 605(e.g., head) having a target dimension 650 (e.g., pupillary distance)and a mobile device 610 positioned in front of a mirrored surface 615 ata distance L. A camera 620 and display 625 are part of the mobile device610, and the display shows a first calibration image 630 above a secondcalibration image 635. For convenience, a reflection 640 of the originalitems 645 is shown as they appear to the camera (“behind” the mirroredsurface 615 at distance L′). The reflected object 605-a and its targetdimension 650-a is seen next to the reflected mobile device 610-a, whichhas a reflected display 625-a and corresponding reflected first andsecond calibration images 630-a, 635-a.

The target dimension 650 in this and other figures is approximatelyin-plane with the display 625. Being at least close to in-plane with thedisplay may allow for simpler calculation of the target dimension 650due to it appearing at approximately the same distance from the camera620 as the calibration images 630, 635. In one example, the object 605may be a face, and the mobile device 610 may be positioned adjacent tothe face so that one or more facial features are at least approximatelyin-plane with the display 625.

In some embodiments, the target dimension 650 may be out of plane withthe display 625, such as being closer to or farther from the mirroredsurface 615 than the display 625. In these embodiments, the distancebetween the mirrored surface 615 and the target dimension 650 may needto be known independent from the distance L so that the target dimension650 may be scaled appropriately (e.g., when it is measured using areference image having a reference dimension on the device 610).

The construction lines between the device 610 and the mirrored surface615 show the path of the light entering the camera 620. The dashedconstruction lines between the mirrored surface 615 and the reflection640 are shown for convenience in understanding the relationship betweenthe original items 645 and the reflection 640. Thus, a construction linerunning from the camera 620 to the reflected target dimension 650-ashows that the camera receives light incident from the original targetdimension 650. The first calibration image 630 and reflected firstcalibration image 630-a have construction lines showing the path oflight from the top and bottom edge of the first calibration image 630 tobeing seen by the camera 620. The second calibration image 635 and itsreflection 635-a also has these top and bottom edge construction linesshown. In this embodiment, the first calibration image 630 is tallerthan the second calibration image 635.

The reflection 640 appears to the camera 620 to be at a distance equalto the sum of L and L′. When the mirrored surface 615 is a 1:1 commonmirror, L is equal to L′, but in other arrangements, such as which themirrored surface is a virtual mirror, the reflection's distance behindthe mirror L′ will change according to the ratio of the mirroredsurface.

FIGS. 7 and 8 expand FIG. 6 to show a diagram 700, 800 having the cameraoutput 705 shown in relation to the original items 645 and thereflection 640. The camera output 705 represents what the camera 620sees when the camera 620 is receiving a view of the reflection 640 whilesimultaneously displaying at least parts of the reflection 640 on thedisplay 625. The reflection 640 includes a reflection of the display(e.g., 625-a), so an infinite-mirror-like effect may be created, similarto when an observer stands between two opposing mirrored surfaces. FIG.8 is an alternate view of FIG. 7 emphasizing the view of the camera 620over the output of the display 625 while depicting the alignment ofcorresponding images (e.g., compare the construction lines betweenimages 630-b and 635 in FIG. 7 against construction lines between images635-a and 630-b in FIG. 8).

The camera output 705 comprises a camera-output object 605-b with acamera-output target dimension 650-b and a camera-output device 610-bwith a display 625-b and first and second calibration images 630-b,635-b. The camera output 705 is shown lower than the original items 645and reflection 640 because the output 705 is being shown on the display625 of the device 610. The display 625 is lower than the camera 620, sothe camera output 705 appears lower to the camera 620 than thereflection 640. If the camera 620 is repositioned with respect to thedisplay 625 (e.g., translated elsewhere on the device 610 or tilted),the apparent location of the camera output 705 changes as well. In thisview, the camera 620 is located and tilted with respect to the display625 so that the camera-output first calibration image 630-b is alignedwith the second calibration image 635 on the display 625, as explainedin further detail below.

Construction lines run from the target dimension 650 to the camera 620to show that target dimension 650 is viewable by the camera 620 in thepositions shown. Corresponding dashed construction lines show how thecamera 620 views the reflected target dimension 650-a and camera-outputtarget dimension 650-b. These lines travel within the bounds of theedges of the displays 625, 625-a, 625-b, so the reflections 650-a, 650-bwill be seen by the camera on the display 625. In FIG. 7, thecamera-output target dimension 650-b is also seen traveling through theupper and lower bounds of the first calibration image 630, so it can bededuced that the first calibration image 630 will obscure the targetdimension 650-b unless at some point the first calibration image 630 isremoved or repositioned.

The camera 620 may simultaneously view the first calibration image 630,via the reflected first calibration image 630-a, and the camera-outputfirst calibration image 635-b, via the display 625. Additionalreproductions of the first calibration image 630 may be viewable if thecamera 620 and device 610 are positioned properly to see reflectionswithin the displayed reflections on the display 625. The camera 620 alsoviews the camera-output target dimension 650-b via the display 625,which may be significant when measuring the camera-output targetdimension 650-b, as described in more detail below.

Dashed construction lines running between the top and bottom of thesecond calibration image 635 and the camera-output first calibrationimage 630-b confirm that, as displayed on the display 625, the firstcalibration image 630-b is aligned with the second calibration image635. In this embodiment, alignment means the top and bottom edges of thefirst calibration image 630-b align with the top and bottom edges of thesecond calibration image 635. With the images aligned in this manner, itcan be concluded that the device 610 is positioned at a predetermineddistance L from the mirrored surface 615. Otherwise, one of the top andbottom edges of the first calibration image 630-b would not meet the topand bottom edges of the second calibration image 635 as seen by thecamera 620 (see also FIGS. 11-13 and their description below).

In some arrangements, the mirrored surface 615 does not provide a 1:1reflection of the original items 645 or the display 625 does not providea 1:1 reproduction of the view of the camera 620, so the size of thereflection 640 or camera output 705 would need to be scaled accordinglyin the diagram 700. For example, the mirrored surface 615 may be avirtual mirror (showing an output of a camera on a display) that hasuneven aspect ratios, pixel size ratios, and the like.

FIG. 9 is an overhead view diagram 900 of a scene comparable to FIG. 8.The original items 905 are reflected by the mirrored surface 615, asshown by the reflection 910 and the camera output 915.

The top of the object 605 (e.g., head) is shown having a targetdimension 920 (e.g., pupillary distance). A target dimension 920 mayinclude two endpoints (e.g., pupils), and a distance between theendpoints. In some arrangements the target dimension 920 may be a spacebetween two points, or the distance from one side of the object to theother. The mobile device 610 having the camera 620 and display 625presents the first calibration image 630 having a first calibrationimage width 925 between left and right edges of the first calibrationimage 630. The second calibration image 635 is also displayed having asecond calibration image width 930 between left and right edges of thesecond calibration image 635. Corresponding elements are found in thereflection 910 (designated by numerals with the letter ‘a’ appended) andin the camera output 915 (designated by numerals with the letter ‘b’appended).

Again, the object 605 is set in front of the mirrored surface 615 at adistance L, and the reflection 910 therefore appears to be at a distanceL′ “behind” the mirrored surface 615. Solid construction lines betweenthe object 605 and mirrored surface 615 illustrate how light isreflected into the camera 620 from the original items 905, and dashedconstruction lines between the mirrored surface 615 and the reflection910 and camera output 915 are for convenience in picturing the view ofthe reflection 910 and camera output 915 as seen by the camera 620.

Accordingly, when the camera 620 is positioned as shown in front of themirrored surface 615, the width of the camera-output first calibrationimage 925-b aligns with the width of the reflection of the secondcalibration image 930-a as seen by the camera 620, giving to the camera620 an appearance of completely overlapping widths of the two images925-b, 930-a. Additionally, the camera 620 views the reflections of thetarget dimension 920-a, 920-b since, as shown by the construction lines,the ends of the target dimension can be seen within the bounds of thedisplays 625-a, 625-b.

The camera output 915 is shown along the same center-line (not shown) asthe reflection 910 and original items 905 because the camera 620 iscentrally positioned on the device 610 in reference to the display 625.If the camera 620 were angled or otherwise changed positions away fromthe center line (not shown), the camera output 915 may be moved inresponse.

Taken together, FIGS. 8 and 9 illustrate how the target dimension 920 isviewable by the camera 620 when the proper depictions of the first andsecond calibration images are properly aligned. FIG. 8 shows howvertical alignment operates, and FIG. 9 shows how horizontal alignmentoperates. For a given size and position of the first and secondcalibration images 630, 635 on the display 625, the distance L may bepredetermined. If the device is moved too close or too far from themirrored surface 615, horizontal or vertical alignment (or both) may notproduce the desired results, since the width and height of thecalibration images may either excessively overlap each other or may fitwithin one another. Conversely, for a given distance L, the relativesize of the calibration images may be calculated as sizes that makealignment at distance L possible.

The distance L may be predetermined to be close enough to the mirroredsurface 615 so that when the device 610 aligns the calibration images,the camera's 620 view of the target dimension 920 (or 920-a, 920-b) maybe large enough to allow effective pattern recognition or more effectiveestimation of the size of the dimension by comparison to a referencedimension on the device 610. For example, in some embodiments, a portion(e.g., an edge length or a distance between two guide points) of thedevice 610 may be used as a reference dimension, so the size and shapeof the calibration images (and distance L) may be predetermined toensure that the camera 620 has a view of that portion of the device whenthe device 610 is aligned, e.g., in accordance with FIGS. 8 and 9. Inanother illustrative example, the distance L may be set to a size thatallows capture of an image of the entire face while the device 610 isproperly aligned. Having an image of the entire face may help manyfacial feature recognition software programs to operate effectively(e.g., helps detect the location of the pupils of the eyes in relationto the rest of the face). Thus, setting up an optimal distance L orscale of calibration images may help ensure that an image of the object605 seen by the camera 620 has a useful size and resolution.

FIG. 10 shows a diagram 1000 of an object 605 (e.g., head) with a targetdimension 920 (e.g., pupillary distance) shown with the mobile device405 having a display 410, camera 415, and showing a first and secondcalibration image 420, 425 on the display 410. The output of the camera415 is not being shown on the display 410, but FIGS. 11-13 illustratehow the display 410 may appear when the output of the camera 415 isbeing shown.

In FIG. 11, the camera output of the first calibration image 420-b isnot aligned with the second calibration image 425 on the display 410.The camera-output first calibration image 420-b is too large to fitwithin the second calibration image 425. In some embodiments, this meansthe device 410 may be too close to the mirrored surface 615. Thus, insome embodiments, the misalignment produces a displayed object 605-bthat may be too large to be measured or incapable of having patternrecognition software find features on the object.

In FIG. 12, the camera output of the first calibration image 420-b isaligned with the second calibration image 425 on the display 410. Theedges of the camera-output first calibration image 420-b align with theedges of the second calibration image 425. In order for this to occur,the device 410 must be positioned at a predetermined distance L that maybe set by the orientation, shape, and size of the first and secondcalibration images 420, 425. In some configurations, this also meansthat the camera-output of the object 605-b may be properly sized andoriented to allow pattern recognition and measurement of the targetdimension 920.

In FIG. 13, the camera output of the first calibration image 420-b isnot aligned with the second calibration image 425 on the display 410.The camera-output first calibration image 420-b may be too small to fitthe second calibration image 425. In some embodiments, this means thedevice 410 may be too far from the mirrored surface 615. Thus, in someembodiments, the misalignment produces a displayed object 605-b that maybe too small to be measured or non-optimal for having patternrecognition software find features on the object.

According to some arrangements, a method may be provided of positioninga mobile device in relation to a mirrored surface wherein the device maybe moved between viewpoints where the camera-output first calibrationimage 420-b is too large (e.g., FIG. 11) or too small (e.g., FIG. 13) toalign with the second calibration image 425 as needed until alignmentmay be reached (e.g., FIG. 12). In some embodiments, the mobile devicemay be positioned adjacent to a portion of the object. In some cases,the mobile device may be positioned adjacent to a facial feature. In thecase where the target dimension is a pupillary distance, the mobiledevice may be positioned at the chin of the face being measured. Ifpositioned at a chin portion of the face, the remaining facial features(e.g., lips, nose, eyes, and/or ears) may be concurrently viewed by thecamera and therefore may be served to a pattern recognition enginecapable of detecting the appropriate features for measurement inrelation to the other features on the face.

Once a mobile device is properly positioned at a mirrored surface (e.g.,as shown in FIGS. 7-9, and 12), an image may be captured by the cameraof the mobile device. This image may be referred to as a referenceimage. A reference image may contain a view of a reference dimension(e.g., the width of the first calibration image or the distance betweenthe second calibration image and the camera) and the target dimension.

FIG. 14 is a diagram 1400 showing a scene where the mobile device 405may be prepared to determine the target dimension 920. In someembodiments, the mobile device 405 creates a reference image from theview of the display 410 at this position. In this embodiment, the firstcalibration image is not displayed at this time so that the camera 415can more fully see the camera-output object 605-b. The camera 415 maystill capture a reference image while the first calibration image isbeing shown, such as, for example, by saving a reference image with thecamera 415 without displaying the first calibration image at the sametime. A reference image may also be obtained by capturing camera outputof the camera-output target dimension 920-b when it is not obscured bythe first calibration image.

The target dimension 920 may be calculated at this point by comparingthe camera-output target dimension 920-b to a reference dimension shownin the display 410. For instance, the camera-output target dimension920-b may be compared to a dimension of the second calibration image 425(e.g., image width 1405). In other examples, the reference dimension maybe a dimension of the camera-output device 405-b (e.g., device width1415) or may be a relative dimension of the camera-output device 405-band an image displayed on the camera-output device 405-b (e.g., distancefrom the camera-output camera 415-b to the center of the camera-outputfirst calibration image 420-b, shown as distance 1410).

In an alternative embodiment, the target dimension 920 may be calculatedfrom a reference image because the reference dimension may be the sizeof the image itself when gathered from the camera. When the device 405is properly positioned for alignment of the first and second calibrationimages, the distance between the device 405 and the mirrored surface(e.g., 615) may be known. When the object 605 (or at least the targetdimension 920) is also at that known distance and the target dimension920 can be detected in the image, a scale for the image can be deduced.For example, at a known distance from a mirror, it may be deduced thatthe bounds of an image of the reflection of the object captured by acamera at that distance extends a certain number of inches in width andheight for objects in-plane with the camera and display (or for a knowndistance in front of or behind the display). Thus, the distance betweentwo points on the object may be calculated by comparing the length ofthe line segment connecting those points to a dimension of the image(e.g., width). In a related embodiment, a grid may be overlaid on thereference image that may be scaled to the known width or height of thereference image, such as a grid showing lines at every inch in thereference image. The target dimension may then be measured against thegrid lines.

The reference dimension may be used to measure the camera-output targetdimension 920-b. For example, a spatial relationship may exist betweenthe two dimensions. In some embodiments, the lengths of the twodimensions may be directly compared in the reference image or on thedisplay 410 of the mobile device 405. For instance, if the width of thesecond calibration image 1405 is a known length, its on-screen size maybe directly compared to the on-screen size of the target dimension 920-band the width of the second calibration image 1405 may be proportionallyscaled upward or downward to produce a measurement for the targetdimension 920.

After measuring the target dimension 920, a 3-D model of the object maybe scaled to match the measurement. For example, when the object is ahead and the target dimension is a pupillary distance, a 3-D model ofthe head may be scaled so that the pupils of the model are a givendistance apart. The rest of the 3-D model may also correspondinglyscale, shrinking or enlarging along with the pupillary distance on the3-D model in accordance with the measured dimension.

FIG. 15 is a flow diagram illustrating an example method 1500 to scale a3-D model using a reflection of a mobile device. In some configurations,the method 1500 may be implemented by the scaling module 115 illustratedin FIGS. 1, 2, and/or 3.

At block 1505, a mobile device and an object having a target dimensionmay be provided at a mirrored surface. The mobile device may have acamera and a display wherein the display may show an output of thecamera. At the mirrored surface, the camera may simultaneously receive areflection of the display and a reflection of the target dimension ofthe object. At block 1510, the display may display first and secondcalibration images.

At block 1515, the display may be positioned so that, as seen by thecamera, the first calibration image aligns with the second calibrationimage in the reflection of the display. At block 1520, the targetdimension of the object may be determined by comparing the targetdimension as seen by the camera to a reference dimension in the outputof the camera.

FIG. 16 is a flow diagram illustrating one example of a method 1600 toscale a 3-D model using a reflection of a mobile device. In someconfigurations, the method 1600 may be implemented by using the scalingmodule 115 illustrated in FIGS. 1, 2, and/or 3.

At block 1605, a mobile device and object may be provided at a mirroredsurface. The mobile device may comprise a camera and a display, wherethe display shows an output of the camera. The camera may simultaneouslyreceive a reflection of the display and a reflection of a targetdimension of the object. In some arrangements, the camera and displaymay be integrally connected, and both the camera and the display facethe mirrored surface while the output of the camera is shown on thedisplay.

At block 1610, the mobile device displays first and second calibrationimages. The first and second calibration images may have a predeterminedsize. For example, the first and second calibration images may have apredetermined widths and heights. The calibration images may be shown ata position on the display where the distance is predetermined between areference point on the device (e.g., a camera) and the calibrationimage. In some cases, one or more calibration images may cover up aportion of the object on the display. In one example, one calibrationimage is a machine-readable identification code such as a QR code, andthe other calibration image is a square block or bracket having theaspect ratio of the QR code (e.g., a square).

At block 1615, the mobile device may be positioned so that thereflection as seen by the camera shows the first calibration imagealigned with the second calibration image. For example, the reflectionof the first calibration image as displayed may overlap with the secondcalibration image on the display or vice versa. In another example, thecalibration images may have interlocking patterns that form a shape whenthe device is properly positioned. Alignment may be determined bywhether the calibration images have a proper scale relationship betweeneach other. In one case, alignment may depend on matching the scale ofthe depiction of the first calibration image as seen by the camera withthe scale of the second calibration image on the display. In anotherembodiment, the calibration images may change appearance when they arealigned. For example, the calibration images may change color when theyare aligned in order to give feedback to the user regarding thealignment status detected by the camera.

At block 1620, the camera captures a reference image containing areference dimension and a target dimension of the object. The targetdimension may be defined by the distance between two points on theobject as seen in the reference image or as seen on the display of themobile device. In some cases, the target dimension may be the distancebetween two shapes on the device, such as, for example, two pupils on aface, in the case of a pupillary distance, or the tips of the ears on ahead. The reference dimension may be defined by a known distance betweentwo points on the device, such as, for example, the distance between twoouter edges of the device, the distance between the camera of the deviceand a calibration image or other image shown on the display, or thewidth of a calibration image or other scaling image shown on thedisplay. The reference image may be captured and stored for processingby the mobile device. In some instances, the reference image may be sentto a server or other computer connected to the mobile device via anetwork for processing measurements and patterns. A reference image maynot be captured in all instances, as the target dimension may bedirectly determined on the display without use of a stored referenceimage if the mobile device has the necessary power and software to doso.

At block 1625, a pattern recognition engine may be implemented to detecta previously-defined feature on the object. The pattern recognitionengine may be pattern recognition software. In some arrangements thepattern recognition engine may be a logic engine or a human-assistedrecognition program. A pattern recognition engine may implement acomputer vision algorithm including a method for acquiring, processing,analyzing, and understanding images. Pattern recognition may includedetection of shapes or features in a still image or moving picture of anobject. For example, the pattern recognition engine may detect the typeof object in an image, such as a face or a corner, or detect anenvironment around an object (e.g., detecting the sky or detecting thehorizon). In some embodiments, the pattern recognition engine may detectsymbols such as handwriting or typed letters. In some embodiments,significant shapes may be detected, such as, for example, squares orstars. A previously-defined feature on the object may be a featurerecognized by the pattern recognition engine. For example, the patternrecognition engine may detect the location of eyes on a face or any ofthe other shapes and features previously mentioned.

The previously-defined feature may have a spatial relationship to thetarget dimension. The spatial relationship may establish that thepreviously-defined feature is part of the target dimension, such as oneor more previously-defined features comprising endpoints of the targetdimension. In one example, the previously-defined features are thepupils of two eyes detected by the pattern recognition engine, and theeyes have a spatial relationship to the target dimension by serving asthe endpoints to a line segment that defines the target dimension.

At block 1630, a target dimension of the object may be determined bycomparing the target dimension as seen by the camera (i.e., thecamera-output target dimension) to a reference dimension as seen by thecamera. The reference dimension may be a predetermined distance betweentwo points in the output of the camera or in a reference image, asdescribed in detail in relation to the above-described configurations.In some embodiments, the lengths of the two dimensions may be directlycompared in the reference image or on the display of the mobile device.For instance, if a width of the second calibration image is a knownlength, its on-screen size may be directly compared to the on-screensize of the target dimension and the width of the second calibrationimage may be proportionally scaled upward or downward to produce ameasurement for the target dimension. In some embodiments, determiningthe target dimension includes obtaining a ratio representing thecomparative length of the target dimension against the referencedimension, then applying that ratio to the known length of the referencedimension to produce the target dimension.

At block 1635, a 3-D model may be scaled using the determined targetdimension. For example, when the object is a head and the targetdimension is a pupillary distance, a 3-D model of the head may be scaledso that the pupils of the model are a given distance apart. The rest ofthe 3-D model may also correspondingly scale by shrinking or enlargingalong with the pupillary distance on the 3-D model in accordance withthe measured dimension. In some arrangements, the determined targetdimension may be used to generate scaling data. The scaling data maycomprise measurement values or other information needed to scale a 3-Dmodel. It may be advantageous to generate scaling data instead ofscaling the 3-D model directly. For example, if block 1635 is performedon a mobile device, the device may not be capable of scaling the 3-Dmodel due to hardware limitations (e.g., limited graphics processingcapabilities), so by generating scaling data, the mobile device maytransfer that function to another machine.

In one example, a scaled 3-D model of the user may be used to renderimages that may be displayed (to the user, for example) on the display.For instance, the scaled 3-D model of the user and a scaled 3-D model ofa product (e.g., a scaled pair of glasses, scaled based on the samescaling standard, for example, based on the user's pupillary distance)may be used to render images for a properly scaled virtual try-on. Inone example, a properly scaled virtual try-on may facilitate a realisticvirtual try-on shopping experience. For instance, a properly scaled usertry-on may allow a pair of glasses to be scaled properly with respect tothe user's face/head. In some cases, this may enable a user to shop forglasses and to see how the user looks in the glasses (via the properlyscaled virtual try-on) simultaneously.

At block 1640, the scaled 3-D model may be displayed. The model may bedisplayed on the display of the mobile device or on another display. Insome embodiments, at block 1640, the scaling data may be sent to aserver (or other computer or mobile device), such as, for example, sentover a network (see FIG. 2 and its attendant description). This may beadvantageous when the server or other computer has more powerfulprocessing ability than the mobile device.

In some embodiments, the method 1600 may be embodied in a softwareprogram or application executable by one or more processors. Forexample, the software program or application may be executed on acomputing device having a processor and electronically associatedmemory, and the processor executes the blocks of the method 1600. Insome arrangements, this means that certain functions may be performed byinstructing a user. For example, the user may be provided instructionsto position the mobile device according to blocks 1605 or 1615. In someembodiments, the user may be required to capture the reference image (asin block 1620). This may entail instructing a user to capture thereference image or instructing the user how to capture a referenceimage.

In some embodiments, a user may position the display according to block1615. The user may be prompted by the application to hold the device inthe recommended position while the application performs other functions.For example, while the user is holding the device still, the applicationmay capture a reference image or change the appearance of thecalibration images. In one case, the application may require the user tohold the device in a position, and one of the calibration images may beremoved from the display, providing more space on the display for otherimages (e.g., the camera output). This may be advantageous since thedevice may be more accurately positioned when using large calibrationimages, but they may be removed once the device is properly positionedand the calibration images are no longer needed for positioning thedevice. However, in some embodiments at least one calibration image isnot removed to allow it to serve as a reference dimension, therebyfacilitating the determination of the target dimension. In anotherexample embodiment, after the user is directed to position the displayto align the calibration images, the position must be maintained for alimited time before a reference image is captured or the targetdimension is determined. If the calibration images leave a state ofalignment as seen by the camera, the user may be instructed to tryagain. This may be beneficial in ensuring that the camera has time tofocus and to ensure that the user is ready for the image to be captured.

FIG. 17 depicts a block diagram of a computer system 1700 suitable forimplementing the present systems and methods. For example, the computersystem 1700 may be suitable for implementing the device 105, 405, 610illustrated in various figures (e.g., FIGS. 1, 2, 4, 5, 6, 7, and 10)and/or the server 210 illustrated in FIG. 2. Computer system 1700includes a bus 1705 which interconnects major subsystems of computersystem 1700, such as a central processor 1710, a system memory 1715(typically RAM, but which may also include ROM, flash RAM, or the like),an input/output controller 1720, an external audio device, such as aspeaker system 1725 via an audio output interface 1730, an externaldevice, such as a display screen 1735 via display adapter 1740, akeyboard 1745 (interfaced with a keyboard controller 1750) (or otherinput device), multiple universal serial bus (USB) devices 1755(interfaced with a USB controller 1760), and a storage interface 1765.Also included are a mouse 1775 (or other point-and-click device)interfaced through a serial port 1780 and a network interface 1785(coupled directly to bus 1705).

Bus 1705 allows data communication between central processor 1710 andsystem memory 1715, which may include read-only memory (ROM) or flashmemory (neither shown), and random access memory (RAM) (not shown), aspreviously noted. The RAM is generally the main memory into which theoperating system and application programs are loaded. The ROM or flashmemory can contain, among other code, the Basic Input-Output system(BIOS) which controls basic hardware operation such as the interactionwith peripheral components or devices. For example, the scaling module115-b to implement the present systems and methods may be stored withinthe system memory 1715. The scaling module 115-b may be one example ofthe scaling module depicted in FIGS. 1, 2, 3, and/or 17. Applications(e.g., application 215) resident with computer system 1700 are generallystored on and accessed via a non-transitory computer readable medium,such as a hard disk drive (e.g., fixed disk 1770) or other storagemedium. Additionally, applications can be in the form of electronicsignals modulated in accordance with the application and datacommunication technology when accessed via interface 1785.

Storage interface 1765, as with the other storage interfaces of computersystem 1700, can connect to a standard computer readable medium forstorage and/or retrieval of information, such as a fixed disk drive1770. Fixed disk drive 1770 may be a part of computer system 1700 or maybe separate and accessed through other interface systems. Networkinterface 1785 may provide a direct connection to a remote server via adirect network link to the Internet via a POP (point of presence).Network interface 1785 may provide such connection using wirelesstechniques, including digital cellular telephone connection, CellularDigital Packet Data (CDPD) connection, digital satellite dataconnection, or the like.

Many other devices or subsystems (not shown) may be connected in asimilar manner (e.g., document scanners, digital cameras, and so on).Conversely, all of the devices shown in FIG. 17 need not be present topractice the present systems and methods. The devices and subsystems canbe interconnected in different ways from that shown in FIG. 17. Theoperation of a computer system such as that shown in FIG. 17 is readilyknown in the art and is not discussed in detail in this application.Code to implement the present disclosure can be stored in anon-transitory computer-readable medium such as one or more of systemmemory 1715 or fixed disk 1770. The operating system provided oncomputer system 1700 may be iOS®, MS-DOS®, MS-WINDOWS®, OS/2®, UNIX®,Linux®, MAC OS X®, or another like operating system.

While the foregoing disclosure sets forth various embodiments usingspecific block diagrams, flowcharts, and examples, each block diagramcomponent, flowchart step, operation, and/or component described and/orillustrated herein may be implemented, individually and/or collectively,using a wide range of hardware, software, or firmware (or anycombination thereof) configurations. In addition, any disclosure ofcomponents contained within other components should be consideredexemplary in nature since many other architectures can be implemented toachieve the same functionality.

The process parameters and sequence of steps described and/orillustrated herein are given by way of example only and can be varied asdesired. For example, while the steps illustrated and/or describedherein may be shown or discussed in a particular order, these steps donot necessarily need to be performed in the order illustrated ordiscussed. The various exemplary methods described and/or illustratedherein may also omit one or more of the steps described or illustratedherein or include additional steps in addition to those disclosed.

Furthermore, while various embodiments have been described and/orillustrated herein in the context of fully functional computing systems,one or more of these exemplary embodiments may be distributed as aprogram product in a variety of forms, regardless of the particular typeof computer-readable media used to actually carry out the distribution.The embodiments disclosed herein may also be implemented using softwaremodules that perform certain tasks. These software modules may includescript, batch, or other executable files that may be stored on acomputer-readable storage medium or in a computing system. In someembodiments, these software modules may configure a computing system toperform one or more of the exemplary embodiments disclosed herein.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the present systems and methods and their practicalapplications, to thereby enable others skilled in the art to bestutilize the present systems and methods and various embodiments withvarious modifications as may be suited to the particular usecontemplated.

Unless otherwise noted, the terms “a” or “an,” as used in thespecification and claims, are to be construed as meaning “at least oneof.” In addition, for ease of use, the words “including” and “having,”as used in the specification and claims, are interchangeable with andhave the same meaning as the word “comprising.” In addition, the term“based on” as used in the specification and the claims is to beconstrued as meaning “based at least upon.”

What is claimed is:
 1. A computer-implemented method for determining atarget dimension of an object, the method comprising: displaying atleast a first calibration image and a second calibration image on adisplay of a mobile device; capturing, via a camera on the mobiledevice, a reflection of an object, a reflection of the first calibrationimage, and a reflection of the second calibration image, the reflectionsbeing produced by a mirrored surface; displaying the capturedreflections of the object, the first calibration image, and the secondcalibration image on the display of the mobile device; capturing, viathe camera of the mobile device, a reflection of the displayed capturedreflections produced by the mirrored surface; and detecting when thecaptured reflection of the displayed captured reflection of the firstcalibration image is positioned relative to the captured reflection ofthe second calibration image.
 2. The method of claim 1, furthercomprising: scaling an image of the object based at least in part on thedetection of the relative positioning of the captured reflections of thefirst and second calibration images.
 3. The method of claim 2, furthercomprising: generating a three dimensional model of the object; andscaling the three dimensional model of the object based on the scalingof the image of the object.
 4. The method of claim 1, furthercomprising: capturing an image of the object in relation to thedetection of the relative positioning of the captured reflections of thefirst and second calibration images.
 5. The method of claim 4, furthercomprising: determining a target dimension of the object from thecaptured image of the object.
 6. The method of claim 5, wherein thetarget dimension is a distance between two points on the object.
 7. Themethod of claim 6, wherein the object is a human face and the display ispositioned relative to the human face.
 8. The method of claim 6, furthercomprising: detecting at least one facial feature of the object, thefacial feature spatially related to the target dimension.
 9. The methodof claim 4, further comprising: relating a detected aspect of the imageto a predetermined reference dimension in relation to the detection ofthe relative positioning of the captured reflections of the first andsecond calibration images.
 10. The method of claim 1, wherein the firstcalibration image is a machine-readable identification code.
 11. Acomputing device configured to determine a target dimension of anobject, comprising: a processor; memory in electronic communication withthe processor; instructions stored in the memory, the instructions beingexecutable by the processor to: display at least a first calibrationimage and a second calibration image on a display of a mobile device;capture a reflection of an object, a reflection of the first calibrationimage, and a reflection of the second calibration image, the reflectionsbeing produced by a mirrored surface; display the captured reflectionsof the object, the first calibration image, and the second calibrationimage on the display of the mobile device; capture a reflection of thedisplayed captured reflections produced by the mirrored surface; anddetect when the captured reflection of the displayed captured reflectionof the first calibration image is positioned relative to the capturedreflection of the second calibration image.
 12. The computing device ofclaim 11, wherein the instructions are executable by the processor to:scale an image of the object based on the detection of the relativepositioning of the captured reflections of the first and secondcalibration images.
 13. The computing device of claim 12, wherein theinstructions are executable by the processor to: generate a threedimensional model of the object; and scale the three dimensional modelof the object based on the scaling of the image of the object.
 14. Thecomputing device of claim 11, wherein the instructions are executable bythe processor to: capture an image of the object in relation to thedetection of the relative positioning of the captured reflections of thefirst and second calibration images.
 15. The computing device of claim14, wherein the instructions are executable by the processor to:determine a target dimension of the object from the captured image ofthe object.
 16. The computing device of claim 15 wherein the targetdimension is a distance between two points on the object, and whereinthe object is a human face and the display is positioned relative to thehuman face.
 17. The computing device of claim 14, wherein theinstructions are executable by the processor to: detect at least onefacial feature of the face, the facial feature spatially related to thetarget dimension.
 18. The computing device of claim 14, wherein theinstructions are executable by the processor to: relate a detectedaspect of the image to a predetermined reference dimension in relationto the detection of the relative positioning of the captured reflectionsof the first and second calibration images.
 19. A computer-programproduct for determining, by a processor, a target dimension of anobject, the computer-program product comprising a non-transitorycomputer-readable medium storing instructions thereon, the instructionsbeing executable by the processor to: display at least a firstcalibration image and a second calibration image on a display of amobile device; capture, via a camera on the mobile device, a reflectionof an object, a reflection of the first calibration image, and areflection of the second calibration image, the reflections beingproduced by a mirrored surface; display the captured reflections of theobject, the first calibration image, and the second calibration image onthe display of the mobile device; capture, via the camera of the mobiledevice, a reflection of the displayed captured reflections produced bythe mirrored surface; and detect when the captured reflection of thedisplayed captured reflection of the first calibration image ispositioned relative to the captured reflection of the second calibrationimage.
 20. The computer-program product of claim 19, wherein theinstructions are executable by the processor to: scale an image of theobject based on the detection of the relative positioning of thecaptured reflections of the first and second calibration images.